Intra-operative poses of the lung of a patient may be computed from 2D live fluoroscopic images. The intra-operative poses can assist a physician in various medical procedures performed in the lung. For example, the pose information may be used to guide an instrument to a region of interest in the lung.
Computing the pose from the live 2D fluoroscopic images, however, is not straightforward. A challenge arises because the live 2D fluoroscopic images are missing information along the projection direction. Hence, the 2D fluoroscopic images need to be aligned with a preoperatively acquired 3D CT to provide volumetric anatomical information, which leads to the 2D-3D rigid registration.
2D-3D rigid registration has been performed using various techniques. See, for example, Otake, Y.; Armand, M.; Armiger, Robert S.; Kutzer, M. D.; Basafa, E.; Kazanzides, P.; Taylor, R. H., Intraoperative Image-based Multiview 2D/3D Registration for Image-Guided Orthopaedic Surgery: Incorporation of Fiducial-Based C-Arm Tracking and GPU-Acceleration, Medical Imaging, IEEE Transactions on, vol. 31, no. 4, pp. 948, 962, April 2012. See also, Gao, Gang et al., Registration of 3D trans-esophageal echocardiography to X-ray fluoroscopy using image-based probe tracking, Medical Image Analysis, Volume 16, Issue 1, 38-49.
A challenge to performing 2D-3D rigid registration, however, is the high number of computations. 2D-3D rigid registration is computationally intense. Various techniques to speedup computations have been attempted. For example, one technique involves accelerating the Digitally Reconstructed Radiograph (DRR). Otake, Y.; Armand, M.; Armiger, Robert S.; Kutzer, M. D.; Basafa, E.; Kazanzides, P.; Taylor, R. H., Intraoperative Image-based Multiview 2D/3D Registration for Image-Guided Orthopaedic Surgery: Incorporation of Fiducial-Based C-Arm Tracking and GPU-Acceleration, Medical Imaging, IEEE Transactions on, vol. 31, no. 4, pp. 948, 962, April 2012. Gao, Gang et al., Registration of 3D trans-esophageal echocardiography to X-ray fluoroscopy using image-based probe tracking, Medical Image Analysis, Volume 16, Issue 1, 38-49
However, despite speeding up the registration, the accuracy established by the rigid 2D-3D rigid registration is reduced because of the patient's non-rigid motion between 3D image acquisition and 2D image acquisition. In particular, the motion of the lung arising from the patient's repositioning, breathing, and beating heart introduces inaccuracies in the rigid registration.
To estimate the lung deformation, 2D-3D deformable registration is needed. For example, Su et al. presented a 2D-3D deformable registration method for lung intervention. (Su P, Yang J, Lu K, Yu N, Wong S T, Xue Z, A fast CT and CT-fluoroscopy registration algorithm with respiratory motion compensation for image-guided lung intervention, IEEE Trans Biomed Eng. 2013 July; 60(7):2034-41, TBME.2013). In Su et al., the deformation in the transversal plan was modeled as a 2D Bspline and the deformation along the z direction was regularized by a smoothing term. Since the control points only move in the 2D plane, this method simplifies the 2D-3D registration to a 2D-2D registration problem.
Khamene et al. performed the non-rigid registration in 2D space for the alignment between the DRR and the fluoroscopic image and then back projected to 3D space. (Ali Khamene, Oliver Fluck, Shmuel Aharon, Deformable 2D-3D Registration, U.S. Pat. No. 8,184,886 B2, Date of patent: May 22, 2012). See also U.S. Pat. No. 8,184,886 to Khamene et al., describing a method for deformable 2D-3D registration.
The above mentioned methods are based on intensity and simplify the 2D-3D deformation problem to a 2D-2D deformation problem. This reduces the degrees of freedom and computation complexity, but undesirably results in two issues: (1) the “overlapping” anatomical structures in 2D image are deformed as one “layer” and therefore the deformation may not capture the “independent” 3D motion of the structures and (2) the 2D-2D deformation model does not allow for a previously occluded/out-of-camera-view structure to appear which may happen as part of an actual 3D deformation. This reduces the accuracy of the 2D-2D deformation approach. Additionally, none of the above described methods use multiple 2D views “simultaneously” to determine a consistent 3D deformation field which gives a more accurate result.
Rivest-Henault et al. presented another 2D-3D deformable registration based on a feature instead of the intensity. The vessel centerlines were automatically extracted from 2D fluoroscopic images and the centerlines served the calculation of the mismatch in the cost function. (Rivest-Henault, D.; Sundar, H.; Cheriet, M., Nonrigid 2D/3D Registration of Coronary Artery Models With Live Fluoroscopy for Guidance of Cardiac Interventions, Medical Imaging, IEEE Transactions on, vol. 31, no. 8, pp. 1557,1572, August 2012). This method appears to address alignment of the blood vessels. However, Rivest-Henault does not provide a solution to the more complicated task, namely, an alignment of the 2D-3D data for an entire or complete lung.
Chou et al. presented another method to perform 2D-3D deformable registration. Chou et al. employed machine learning to build the mapping from the 3D deformation profile to the 2D DRR in the training stage, and then based on the distance between the test DRR and the training DRR, weighted the deformation. (Chen-Rui Chou, Stephen Pizer, Real-Time 2d/3d deformable registration using metric learning, MCV'12 Proceedings of the Second international conference on Medical Computer Vision: recognition techniques and applications in medical imaging, Pages 1-10.)
Others have described alternative registration techniques including registration of images based on use of a graphics processing unit (GPU). GPU-based processing is known to speed up the time for performing computations. Examples of GPU-based registration are described in U.S. Pat. Nos. 7,483,034 and 7,706,633, both to Chefd'hotel et al. However, in the case of a GPU-based registration, parallelizing the DRR is not enough to speed up the process. Parallelizing the DRR may only result in reducing computational time incrementally, not substantially for 2D-3D deformable registration.
Notwithstanding the above, it is still desirable to develop improved GPU-based registrations techniques. Improved techniques that can perform an accurate 2D to 3D deformable registration on the entire body organ and, at the same time, substantially reduce the computational time of 2D-3D deformable registration.